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Commissioner's Foreword 



Dear Superintendents, Principals, Charter School Leaders, Directors of Collaborates and Chapter 
766 Private Schools, and Other Interested Parties: 

This guide is intended to further explain results and other information contained within the Test 
Item Analysis Report , the School Report, and the District Report for the Massachusetts 
Comprehensive Assessment System (MCAS) tests of spring 1999. We hope that you will find it useful 
as you review the results for your school and/or district. 

The MCAS tests are an important part of our state’s effort to improve teaching and learning 
across the Commonwealth. It is vital that all teachers, administrators, and parents understand the 
MCAS testing program, as all have essential roles to play in helping students to succeed. The state 
is committed to working in partnership with schools and communities to improve the academic 
achievement of all public school students in Massachusetts. 

If you have questions after reviewing this guide, the Test Item Analysis Report , the School Report , 
or the District Report , please contact MCAS Support Services at 1-800-737-5103. 

Sincerely, 




David P. Driscoll 
Commissioner of Education 
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I. Document Purpose 



The primary purpose of this document is to support local educators’ use of reports of results from 
the spring 1999 administration of the Massachusetts Comprehensive Assessment System (MCAS). 
This document describes and explains how to interpret information provided in the following 
MCAS reports: 

• MCAS Tests of Spring 1999 : Test Item Analysis Report 1 

• MCAS Tests of Spring 1999 ; School Report 

• MCAS Tests of Spring 1999: District Report 

These reports contain information that will be valuable to schools and districts in their efforts to 
better serve the academic needs of individual students and to evaluate and improve curriculum and 
instruction. In addition, this document can help school and district personnel communicate with 
their communities about the MCAS test results. 



1 The MCAS Test Item Analysis Report contains confidential test results for individual students. Therefore, the 
Test Item Analysis Report is a nonpublic document subject to access restrictions dictated by state and federal 
law. See page 3 for additional details. 
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II. General Guidelines for the 
Interpretation and Use of MCAS Reports 

ALIGNMENT OF CURRICULUM AND INSTRUCTION 

Test results contained in the 1999 reports should be interpreted with caution. At the time these 
MCAS tests were administered, many schools were still in the process of aligning their curricula 
with the new state standards defined in the Massachusetts Curriculum Frameworks. As a result, 
some students may not have been exposed to all of the content covered by the MCAS tests. Also, 
because the tests were based on the high academic standards in the Frameworks , the tests were 
likely very challenging for many students. Over time, as students and teachers become more familiar 
with the new standards, test results should show student progress. 

USE OF MCAS STUDENT-LEVEL RESULTS 

MCAS results are intended to evaluate how well students and schools are achieving the learning 
standards contained in the Curriculum Frameworks. Individual student results and school-level 
results will help teachers and parents focus on areas in which students need assistance in order to 
improve their individual performance. It is important to note that MCAS is only one indicator of 
student performance and, therefore, should not be used as the sole basis for referring students to 
special education. 

MULTIPLE DATA POINTS NEEDED FOR TREND ANALYSIS 

Two years’ test results provide limited information about a school’s or district’s overall performance. 
As with any evaluation, school and district test results are most meaningful when compared with 
other indicators and when examined over several years for long-term trends in student performance. 

STATE REGULATIONS REGARDING CONFIDENTIALITY OF STUDENT RECORDS 

The Massachusetts Student Record Regulations (603 CMR 23.00) and the Family Educational Rights 
and Privacy Act (FERPA) require that access to individual student results, including those provided 
in the MCAS Test Item Analysis Report and the MCAS Student Report for Parents/Guardians , be 
restricted to the student, the student’s parents/guardians, and authorized school personnel. 
Superintendents and principals are responsible for maintaining the privacy and security of all student 
records. In accordance with these state and federal regulations, authorized school personnel shall 
have access to the student records of students to whom they are providing services, only when such 
access is required in the performance of their official duties. 

MAKING COMPARISONS AMONG STUDENTS, SCHOOLS, AND DISTRICTS 

Scaled scores are the most appropriate statistic to use when making comparisons of MCAS results 
among students, schools, and districts. When interpreting the meaning of these comparisons, how- 
ever, it is important that decision-makers — teachers, administrators, and policy makers — recognize 
that some apparent differences in scaled scores may not be statistically or educationally significant, 
some guidelines for comparing results are explained on the next page. 
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Comparisons of Student-Level Scores 

When comparing the scores within a subject area and grade level test of two or more students, or 
comparing the scores within a subject area and grade level test of a student to that of the school, 
district, or state, it is critical to examine the probable range of the student’s scores displayed in the 
Student Report for Parents/Guardians. If there is an overlap in the probable ranges of the scores 
being compared, there is little statistically or educationally significant difference between the two 
scores. This is true even if the two scores fall on the opposite sides of a performance level threshold. 



Comparisons of School- ond District-Level Scores 

The statistical significance of these comparisons is based on the number of students tested. Whether 
making comparisons within a single year or across years, the table below will assist in comparing 
the scores of two or more schools or districts, comparing the scores of a school to the district and/or 
state, or the scores of a district to the state. The table shows the smallest differences in scores that 
represent a statistically significant difference in performance, based on the number of students tested 
in the school and/or district. 

It is not appropriate or possible to compare a school’s or district’s scaled scores across two subject 
areas since the scaled scores in each subject area were determined by separate standard-setting 
processes. 



Table 1. Number of Scaled Scores Points Denoting Minimally 
Statistically Significant Differences for School/District Results 



Number of Students Tested in School/District 


Grade 


Subject 


10 


25 


50 


100 


200 




English Language Arts 


3 


2 


2 


1 


1 


4 


Mathematics 


6 


4 


2 


2 


1 




Science & Technology 


5 


3 


2 


1 


1 




English Language Arts 


5 


3 


2 


1 


1 


8 


Mathematics 


7 


4 


3 


2 


2 




Science & Technology 


6 


4 


3 


2 


1 




History and Social Science 


5 


3 


2 


1 


1 




English Language Arts 


4 


3 


2 


1 


1 


10 


Mathematics 


7 


4 


3 


2 


1 




Science & Technology 


5 


3 


2 


2 


1 
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OVERVIEW 



III. Understanding the 
Test Item Analysis Report 



The Test Item Analysis Report provides results from the MCAS tests of spring 1999 for students test- 
ed in grades 4, 8, and 10. This MCAS report shows how each student tested in a school 

• answered every common multiple-choice question 

• performed on each common open-response and short-answer question 

• performed on the English Language Arts Composition (writing prompt) 

The report also summarizes overall performance at the school, district, and state levels for each of 
these items. Each school receives a separate Test Item Analysis Report for each subject area and 
grade tested in the school. Reports for English Language Arts differ slightly in structure from those 
for Mathematics, Science & Technology, and History and Social Science (see pages 13-18 of this 
chapter for details). 

This detailed information at the student, school, and district levels can be used for 

• identifying strengths and weaknesses exhibited by students 

• curriculum evaluation and planning in the classroom, school, and district 

• summary reporting to parents and the public (See caution regarding confidentiality 
of student records below). 

As noted above, the Test Item Analysis Report shows results for MCAS common items, the basis for 
all student-level MCAS scores. The report is designed to be used in conjunction with the publication, 
The Massachusetts Comprehensive Assessment System: Release of Spring 1999 Test Items , which 
shows all the questions upon which the Test Item Analysis Report is based. When the report and the 
publication are used together, educators are provided with a detailed picture of student performance. 
This picture should be used in combination with other information from the classroom to develop a 
broad-based overview of students’ academic achievement. 

Caution: Confidentiality of Student Records. The Massachusetts Student Record Regulations (603 
CMR 23.00) and the federal Family Educational Rights and Privacy Act (FERPA) require that access 
to individual student results, including those provided in the MCAS Test Item Analysis Report , be 
restricted to the student, his/her parents/guardians, and authorized school personnel. Since the Test 
Item Analysis Report provides individual student level results for all students tested in a particular 
school, schools must not provide parents/guardians with direct access to the school’s report. To facili- 
tate the sharing of individual student results contained in the Test Item Analysis Report with parents/ 
guardians, individual student labels are provided for each student. Each student label contains all of 
their student-level results included on the Test Item Analysis Report. 

Superintendents and principals are responsible for maintaining the privacy and security of all student 
records. In accordance with these state and federal regulations, authorized school personnel shall 
have access to the student records of students to whom they are providing services, only when such 
access is required in the performance of their official duties. 
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A. English Language Arts Test Item Analysis Report 




ERjt 
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A. English Language Arts Test Item. Analysis Report 



INDIVIDUAL STUDENT RESULTS 

Individual student results for all common English Language Arts test items are provided in the first 
section of this report. The number of pages that comprise this section will depend upon the total 
number of students at the tested grade. 




Massachusetts Comprehensive Assessment System 
MCAS Tests of Spring 1999 
Test Item Analysis Report 

English Language Arts 



See final page for the legend for codes and scores . 



School: Fictitious School 



School ID#: 001001 



District: Fictitious District 



Grade: 8 



Page: 



SCORER COMMENTS 

DJ - details are not used effect r/ety 

DK a delate need to be developed instead of just listed 

OX = details are carefiity chosen and relevant 

DY » details support the topic 

GUP 2 grammatical rules era applied correctly 

GUO = writing shows control of vocabulary and word 



GUR 2 grammatical rites are not applied correctly 
GUS = words are not always used oonectty 
L) = fonited vanety of words used 
LK = sentence structure needs more variety 
LX 2 word choice makes the writing more meaningful 
LY = language used creates a distinctive voice, tone, or 
style 



= mechanics are used correctly and in complicated 
sentences 

= mechanics are not used well 
= student needs to proofread more carefuBy 
» ideas need to be better organized 
= shifts between ideas are weak 
= writing shows evidence of planning 

isterrUv focused from beginning lo end 



* sentence structure is coned 
= sentence structure is not coned 
= response needs to be more appropriate 

to the task's requiements 
2 topic needs more development 
= topic or idea development is effective 
and appropriate 

* topic or idea development is original 




/. Scorer Comments: Codes Used to Indicate Writing Performance 

In addition to receiving a score on the English Language Arts Composition test, each student’s 
writing performance received a minimum of two, and a maximum of four, comments about strengths 
or areas needing improvement in the student’s writing. 

The box at the top of the English Language Arts Test Item Analysis Report labeled Scorer Comments 
(Section 1) lists the two- or three-letter codes used by scorers to indicate comments on a student’s 
writing performance, e.g., “TX=topic or idea development is effective and appropriate on the English 
Language Arts composition test.” Results according to these codes appear in the columns marked 
SC. 



w 




H 




Z 


E 


LLI 

5 


3 


5 


in 


O 

O 


c 

E 

o 


CL 

ID 


8 


CL 


o 


O 


w 


O 


c 


W 








mtm 



SCORER COMMENTS 

DJ = details are not used effectively 
DK = details need to be developed instead of just listed 
DX = details are carefully chosen and relevant 
DY = details support the topic 
GUP = grammatical rules are applied correctly 
GUQ = writing shows control of vocabulary and word 
usage 



GUR = grammatical rules are not applied correctly 
GUS = words are not always used correctly 
LJ = limited variety of words used 
LK = sentence structure needs more variety 
LX = word choice makes the writing more meaningful 

LY = language used creates a distinctive voice, tone, or 
style 

MP = writing is clearer because mechanics are used well 
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2. Item Information 









Item Number 


1 


1 


1 


1 


1 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


14 


15 


16 


17 


18 


19 


20 


21 


221 




Item Type 


WP 


WP 


WP 


WP 


WP 


WP 


MC 


MC 


MC 


MC 


MC 


MC 


MC 


MC 


OR 


MC 


MC 


MC 


MC 


MC 


MC 


MC 


MC 


OR 


MC 


MC 


My 




Correct MC Answer 














A 


D 


C 


A 


B 


B 


A 


D 




D 


B 


C 


A 


B 


C 


D 


B 




D 


A 






Names Reporting Category 

— ■ 


CT 


CC 


sc 


sc 


sc 

-SB- 


sc 


LT 


LT 


LT 


LA 


LT 


LT 


LT 


LT 


LT 


LA 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LA 





The graphic above (Section 2) shows columns that contain four types of information — item number, 
item type, correct multiple-choice answer, and reporting category — for each MCAS English Language 
Arts common item. 



Item Number corresponds to the number of the question as it appears in the Release of Spring 1999 
Test Items publication. Note that item number 1 on all 1999 Test Items Analysts Reports (grade 4, 
grade 8, and grade 10) is repeated across six columns. These columns show the scores and comments 
for the Composition: a numeric score for topic development (CT); a numeric score for writing 
conventions (CC); and up to four scorer comments about the student’s writing performance (SC). 

See below (Body of the Report) for further explanation about the data in these columns and their 
interpretation. 

Item Type indicates whether the item was a multiple-choice (MC) or open-response (OR) question, 
or a writing prompt (WP). 

Correct MC Answer provides the letter option (A, B, C, D) for the correct answer for multiple- 
choice questions. 

Reporting Category indicates the sub-area in English Language Arts in which each question is report- 
ed in the School and District Reports. Reporting category codes are explained in the Legend (details 
on this section are provided on page 12 of this document) on the final page of the Test Item Analysis 
Report. A complete list of all MCAS reporting categories is provided in Appendix B of this docu- 
ment. Columns containing scorer comments are indicated under the reporting category code SC. 



3. Body of the Report 





Names Reporting Category 


CT 


CC 


sc 


sc 


sc 


SC 


LT 


LT 


LT 


LA 


LT 


LT 


LT 


LT 


LT 


LA 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LT 


LA 


7 




1. 


LASTNAME079, FIRSTNA 


9 


7 


TX 


DX 


SR 












































( 




2. 


LASTNAME047, FIRSTNA 


11 


6 


TY 


OX 


MS 












































\ 




3. 


LASTNAME087, FIRSTNA 


NS 


NS 










D 


A 


A 


+ 


+ 


A 


B 


A 


2 


A 


+ 


+ 


B 


C 


+ 


+ 


+ 


2 


C 


D 


a\ 




4. 


LASTNAME014, FIRSTNA 


NS 


NS 










D 


«► 


A 


D 


D 


C 


+ 


j3 


J2. 


A 


+ 


+ 


D 


■f 


■f 


■f 


A 


2 


C 




D 




5. 


LASTNAME048, FIRSTNA 
LASTNAM^^ — 


6 

-oJ 


6 


SP 


TJ 






B 

-CX- 


C 


A 


C 


+ 


+ 


+ 

r 


B 

-C- 


0 

_a_ 


A 


C 


B 


+ 


C 

Lq_ 


+ 

-a. 


A 


+ 


0 


B 


+ 


+ 



Individual Student Results 



Section 3 lists all students in the tested grade alphabetically by last name. Individual student results 
are reported in alternately shaded rows across multiple columns. This section contains 
a row for all students enrolled in the school on the first day of testing. This number was provided by 
each school principal on the Principal’s Certification of Proper Test Administration form. Enrolled 
students for whom no answer booklet was returned are indicated by a blank row. These students are 
classified as Absent and receive a minimum scaled score of 200. For students who are not included in 
the computation of aggregate school- and district-level results (see page 21 of this document), indi- 
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vidual item results are provided, but no scaled score or performance level is reported. Where the scaled score 
would be reported, there will be a two- or three-letter code used to identify the reason why this student was not 
included in the aggregate results. These codes appear in the Legend on the final page of the report. A small number of 
students may receive a code of “INC” signifying that there was incomplete data (e.g., the student’s answer 
booklet and composition could not be matched, or the student responded in more than one answer booklet), 
and, therefore, no results could be calculated. 

For multiple-choice questions, 

• a + sign in any MC column indicates the student answered the question correctly; 

• a letter indicates the incorrect option (A, B, C or D) that was selected; 

• a blank indicates the student did not answer the question. 

Open-response questions were scored on a 0 through 4 scale; the student’s scores for open-response questions 
are shown in the OR columns. 

Writing scores and scorer comments for students’ compositions are indicated by numbers or codes in the six 
writing prompt (WP) columns (item 1). Each student’s composition was scored independently by two trained 
scorers. Each scorer gave the composition two separate scores: 

• a score from 1-6 in topic development (CT), e.g., writing development, organization, use of detail, variety 
in sentence structure and language; and 

• a score from 1-4 for the student’s use of standard English writing conventions (CC), e.g., grammar, punc- 
tuation, mechanics of writing. 

The student’s score as shown in each of these two columns (CT and CC) represents the total of the points 
awarded in each area by both scorers. Any student who wrote a totally illegible composition, wrote an off-topic 
response, or who wrote in a language other than English was given a notation of NS (not scoreable), which is 
shown as NS in this report. Zero points were awarded in such cases. 

The first scorer was required to note two comments about the student’s writing: one in the area of topic 
development and one in the area of conventions. The second scorer could choose one of the following options 
in commenting about the student’s writing: 

• note two strengths, one in each scoring area; or 

• note a strength in one scoring area and an area needing improvement in the other 
scoring area; or 

• note one area needing improvement in either scoring area. 

These scorer comment codes are shown in the SC columns and defined on page 7 of this document. 



Summary Scores 

The final three columns of each row contain summary information about the student’s performance. 

The student’s Total Score is shown in the column following the final item, reported in terms of “raw” score 
points attained in English Language Arts. The student’s Total Score was calculated by adding one point for 
each correct multiple-choice (MC) question to the student’s scores for each open-response (OR) question 
and the Composition (CT and CC). 



0 
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The student’s scaled score is shown in the next column to the right, reported on a scale ranging from 200 to 280. 

This scaled score translates to one of the four MCAS performance levels, which is indicated in the final column. 
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RESULTS AGGREGATED AT THE SCHOOL, DISTRICT, AND STATE LEVELS 

The final page of the English Language Arts Test Item Analysis Report (sample below) shows the 
performance of students on common test items aggregated at school, district, and state levels. 




"53T" 



Massachusetts Comprehensive 
A ssessment System 
MCAS Tesls of Spring 1999 
Test Hem Analysis Report 

English Arte 
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J. Percentage of Correct Responses (M()/Averoge Score (OR ond WP) 





Percentage of Correct Responses (MC)/ \ 
Average Score (OR and WP) \ 




Item 

Number 


Item 

Type 


Raporffftg 

Category 


School 


District 


State tj 




1 


WP 


CT 


7.72 


7.33 


8.11 \ 




1 


WP 


CC 


5.07 


4.82 


5.11 \ 




2 


MC 


LT 


7.72 


7.33 


8.11 / 




3 


MC 


LT 


5.07 


4.82 


5.11 / 




4^J 




LT 




L --Z43 


6M1JJ 



This section shows aggregated data for the school, district, and state. Items are listed by item num- 
ber, type, and reporting category, as explained earlier in this chapter. The table shows, for the school, 
district, and state, the percentage of correct responses for each multiple-choice question and the average 
score for each open-response question and writing prompt. These statistics will help educators compare 
the performance of students in the school to students in the district and across Massachusetts, and 
identify strengths and weaknesses in local curriculum and instruction. 



2. Percentage of School's Totol Student Responses 




This section shows two kinds of aggregated data: 

• For each multiple-choice question, this section shows the percentage of students in the school 
who chose each answer option (blank, A, B, C, or D); the correct answer is provided in the far 
right column. This information will be most useful to educators in analyzing students’ mis- 
takes and possible misconceptions held by students in the school. 

• For open-response questions, the section shows the percentage of students in the school at 
each score point (blank/0, 1, 2, 3, or 4). 
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3. Legend 

This section is provided as a quick reference for anyone reviewing the 
report: it explains and gives additional information on how to interpret 
codes that appear throughout the report pages. 

The information on Item Numbers and Item Type applies to both the first 
section of the report and this final page. 

Reporting Category information given here refers to the two- and three- 
letter codes that appear in columns (above the horizontal line) within the 
first section of the report. A complete list of MCAS reporting categories is 
provided in Appendix B of this document. 

The information given in the Key to Score Codes section explains the sym- 
bols, letters, and numbers (also in the first section of the report) which 
appear in the columns across from each student’s name. 
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I ho MCAS tfWLJTOfA. ffnfcvjce of Spring i P99 
TYiSJ NSmi 

hem Type 

MG • M^dptochdoii bom 
OR n Open* restore* (tern 
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blank space * no snswer gherc 
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B. Mathematics, Science Sc Technology, and History and 

Social Science Test Item Analysis Reports <2 

The formats of the Mathematics, Science 8c Technology, and History and Social Science Test Item 
Analysis Report are identical. The following guidelines for interpretation of the report sections apply 
to all three of these reports. 
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INDIVIDUAL STUDENT RESULTS 



Individual student results for all common Mathematics, Science & Technology, and History 
and Social Science test items are provided in the first section of the Test Item Analysis Report. 2 
The number of pages included in this section will depend upon the total number of students 
at the tested grade. 




Massachusetts Comprehensive Assessment System 
MCAS Tests of Spring 1999 
Test Item Analysis Report 

Mathematics 
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School ID#: 001001 



District: Fictitious District 



Grade: 8 



Page: i 
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I. Item Information 
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Individual student results are reported in alternately shaded rows across multiple columns. Above, 
Each column also contains four types of information — item number, item type, correct multiple- 
choice answer, and reporting category — for each common item. 



Item Number corresponds to the number of the question as it appears in the Release of Spring 1999 
Test Items publication. 



Item Type indicates whether the item was a multiple-choice (MC), open-response (OR), or short- 
answer [(SA) used in Mathematics only] question. 



Correct MC Answer provides the letter option (A, B, C, D) for the correct answer for multiple- 
choice questions. 



2 The results from one common question administered in grade 4 Science & Technology in Spring 1999 
were not used in determining individual student scores for students who took the large-print version of MCAS. 
Because of the content of this question, it could not be included in the large-print version of the test. Test 
scores for students taking the large-print version were statistically adjusted to compensate for the different 
number of items contributing to their total scores. 



14 



The Massachusetts Comprehensive Assessment System: 
Guide to Interpreting the 1999 MCAS Reports for Schools and Districts 




Reporting Category indicates the sub-area in either Mathematics, Science & Technology or History and 
Social Science in which each question is reported in the School and District Reports. Reporting category 
codes are listed in the Legend on the final page of the Test Item Analysis Report. 



2.Body of the Report 
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Individual Student Results 



The body of the report lists all students in the tested grade alphabetically by last name. Individual student 
results are reported in alternately shaded rows across multiple columns. This section contains a row for all 
students enrolled in the school on the first day of testing. This number was provided by each school prin- 
cipal on the Principal's Certification of Proper Test Administration form. Enrolled students for whom no 
answer booklet was returned are indicated by a blank row. These students were classified as Absent and 
received a minimum scaled score of 200. For students who are not included in the computation of aggre- 
gate school- and district-level results (see page 21 of this document), individual item results are provided, 
but no scaled score or performance level is reported. Where the scaled score would be reported, there will 
be a two- or three-letter code used to identify the reason why this student was not included in the aggre- 
gate results. These codes appear in the Legend. A small number of students may receive a code of “INC” 
signifying that there was incomplete data (e.g., the student’s answer booklet and composition could not be 
matched, or the student responded in more than one answer booklet). 

For multiple-choice questions, 

• a + sign in any MC column indicates the student answered the question correctly; 

• a letter indicates that an incorrect option (A, B, C or D) was selected; 

• a blank indicates the student did not answer the question. 

Open-response questions were scored on a 0 through 4 scale; the student’s scores for open-response 
questions are shown in the OR columns. 

Short-answer questions were used in Mathematics only and were scored on a 0 through 1 scale; the 
student’s scores for short-answer questions are shown in the SA columns only on the Mathematics Test 
Item Analysis Report. 



Summary Scores 

The final three columns of each row contain summary information about the student’s performance 
The student’s Total Score is shown in the column following the final item, reported in terms of “raw” 
score points attained in Mathematics, Science & Technology, or History and Social Science. The student’s 
Total Score was calculated by adding one point for each correct multiple-choice (MC) question to the student’s 
total scores for each open-response (OR) question and, in Mathematics, each short-answer (SA) question. 
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RESULTS AGGREGATED AT THE SCHOOL, DISTRICT, AND STATE LEVELS 

The final page of the Test Item Analysis Report (sample of Mathematics version shown below) 
summarizes the performance of students on common test items aggregated by school, district, and 
state. In 1999, Grade 10 History and Social Science results are reported only at the student level. 
No aggregate school, district, and state results are reported. 
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ERIC 




Massachusetts Comprehensive 
Assessment System 
MCAS Tests of Spring 1999 
Text Item Analysts Report 
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The student’s scaled score is shown in the next column to the right, reported on a scaled ranging 
from 200 to 280. This scaled score translates to one of the performance levels indicated in the final 
column. Note: In 1999, no scaled score or performance level is reported on the Grade 10 History 
and Social Science test. 

/. Percentage of Correct Responses (M()/Averoge Score (SA and OR in Mathematics; OR in Science 
& Technology, History and Social Science) 

O 




This section shows aggregated data for the school, district, and state. Items are listed by number, 
type, and reporting category, as explained earlier in this chapter. This section shows, for the school, 
district, and state, the percentage of correct responses for each multiple-choice question and the 
average score for each open-response question and, for Mathematics, each short-answer question. 
These statistics allow educators to compare the performance of students in the school to students in 
the district and across Massachusetts, and identify strengths and weaknesses in curriculum and 
instruction. 



2. Percentage of School's Totol Student Responses 




This section shows two kinds of aggregated data: 

• For each multiple-choice question, this section shows the percentage of students in the school 
who chose each answer option (blank, A, B, C, or D); the correct answer is provided in the 
far right column. This information allows educators to analyze students’ mistakes to identify 
possible misconceptions held by students in the school. 
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• For open-response and short-answer (in Mathematics only) ques- 
tions, the section shows the percentage of students in the school at 
each score point (blank/0, 1, 2, 3, or 4 for open-response questions; 
blank/0 or 1 for short-answer questions). 

3. Legend 

This section explains how to interpret codes that appear throughout the 
report pages. 

The information on Item Numbers and Item Type applies to both the first 
section of the report and this final page. 

Reporting Category information given here refers to the two-letter codes 
that appear in columns (above the horizontal line) within the first section 
of the report. A complete list of MCAS reporting categories is provided in 
Appendix B of this document. 

The information given in the Key to Score Codes section explains the 
symbols, letters, and numbers used to report student’s performance. 
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IV. Understanding the 
School and District Reports 



OVERVIEW 

The School Report and the District Report each provide MCAS results for schools/districts based on 
the testing of local students in grades 4, 8, or 10. A separate School Report and District Report has 
been produced for each grade level — 4, 8, or 10 — tested in spring 1999. 

Text in this chapter refers only to the MCAS School Report. The data which are reported, the report 
format, and guidelines for interpreting the reported data are identical for the School Report and the 
District Report. The only difference between the School Report and the District Report is that the 
District Report includes no individual school data. Educators and others who are reviewing the 
District Report should refer to this chapter for applicable information. 



IDENTIFICATION 




School: Fictitious School 



School ID#: 001001 



District: Fictitious District 
Grade: 8 



The box in the upper right corner of each page shows the school name, its six-digit identification 
number, the district name, and the grade level for which the report has been generated. 

BASIS FOR RESULTS 

With the exception of subject area subscores, all results in the MCAS School Report are based solely 
on common items (the set of items answered by all students), and represent the aggregate of individ- 
ual student scores (performance level results and scaled scores). See page 26 of this chapter for infor- 
mation on interpreting subject area subscores. 

MINIMUM NUMBER OF STUDENTS NEEDED TO GENERATE REPORTS 

In order to ensure confidentiality of individual student results and discourage generalizations about 
school performance based on very small populations, the Department of Education has established 
minimum numbers of students tested for reporting results for 

• overall school averages 

• school averages disaggregated by student status 

• subject area subscores 
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Consequently, schools with a very small number of students enrolled in a grade tested may not show 
results in some sections of their School Report. No School Report is generated for any school that 
tested fewer than ten students at a particular grade; results for students in these schools are included 
in district- and/or state-level results. The minimum number of tested students required for each 
reporting section is presented in the table below. 



Table 2. Minimum Number of Students Tested (at Each Grade) 



to Generate School Results 


Reporting Section 


Minimum Number 




of Students 


Percentages of Students by Performance Level and Scaled Scores 


10 


Performance Level and Scaled Score Results Disaggregated by Student Status 10 


Subject Area Subscores 3 


12 



ORGANIZATION OF SCHOOL REPORT 

Each School Report includes the following categories of results from the spring 1999 MCAS tests: 

A. Students Tested 

B. Performance Level Results 

C. Distribution of Results by Scaled Score Interval 

D. Results by Student Status 

E. Subject Area Subscores 

F. Three- Year Comparison of School Results 



3 All twelve forms of the test must hove been administered in a school for accurate computation of subject area 
subscores. If fewer than the 12 forms of the test were administered in a school, subject area subscores were 
not produced for that school. 
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A. STUDENTS TESTED 

(Report pages 3-5 for grades 4 & 10; pages 3-6 for grade 8) 



The pages titled Students Tested show, for the school, district, and state, the number and percentage of students 
in each of three student status groups included/not included in scores in the spring 1999 MCAS tests. 

Data for student participation by subject area can be found on the following pages of the Report: 

• Page 3: Student participation in English Language Arts 

• Page 4: Student participation in Mathematics 

• Page 5: Student participation in Science Sc Technology 

• Page 6: Student participation in History and Social Science (Grade 8 only) 

For each of the above, data for students included/not included in scores at the school level are reported 
according to the following student status groups (these definitions are also found on page 2 of the 
Report): 

• Regular: Students who did not meet the definitions below for Students with Disabilities or Limited 
English Proficient (LEP) students as of spring 1999. 

• Students with Disabilities: Students who had an Individualized Education Plan (IEP) or were 
provided with a plan of instructional accommodations under Section 504 of the Rehabilitation 
Act of 1973 as of spring 1999. 

• Limited English Proficient: Limited English Proficient students were defined as students who, in 
spring 1999, were: 

(1) enrolled in a Transitional Bilingual Education (TBE) program or received English as a 
Second Language (ESL) support; OR 

(2) not born in the United States, whose native language is a language other than English, 
and who are currently not able to perform ordinary classroom work in English; OR 

(3) born in the United States to non-English speaking parents and who are currently not able 
to perform ordinary classroom work in English, 

The table below is an example of those included on pages 3 to 6 of the School and District Reports . 



Table 3: Sample Table Showing Students Included in Scores and Not Included in Scores 



STATUS 


NUMBER 

OF 

STUDENTS 

ENROLLED 


INCLUDED IN 
SCORES 


NOT INCLUDED IN 
SCORES 


TES1 

U 


“ED 

% 


ABSE 

# 


:NT 

% 


MEDICAL 

DOCUMEN 

# 


.LY 

ITED 

% 


OTHE 
NOT TES 

# 


R 

TED 

% 


REGULAR 


175 


170 


97 


5 


3 


0 


0 


0 


0 


STUDENTS WITH 
DISABILITIES 


20 


18 


90 


1 


5 


0 


0 


1 


5 


LIMITED ENGUSH 
PROFICIENT 


5 


3 


60 


0 


0 


0 


0 


2 


40 


TOTAL 


200 


191 


96 


6 


3 


0 


0 


3 


1 
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Number of Students Enrolled 



Column 2 of each table shows the number of students Enrolled in the tested grade on the first day of test- 
ing. The total number of students Enrolled is comprised of the number of students Tested or Absent plus 
the number of students that were Medically Documented Absent and Other Not Tested. The number of 
students reported as enrolled in English Language Arts on page 3 of the School Report may differ from 
the corresponding numbers for other subject areas on pages 4-6 because the Composition portion of the 
English Language Arts test was administered during a separate testing period. This enrollment informa- 
tion was provided by each school principal on the Principal’s Certification of Proper Test Administration 
form. 



Students Included in Scores 

Columns 3 through 6 of each table show the number and percentage of students who were included in 
the 1999 MCAS school results. 

Tested Students 

The numbers and percentages of Regular students include students who do not meet the definition of 
Students with Disabilities or Limited English Proficient (LEP) students and who participated under stan- 
dard conditions. 

The numbers and percentages of Students with Disabilities include students who participated under 
standard conditions, as well as students who participated with approved accommodations in accordance 
with Department of Education guidelines. 

The numbers and percentages of Limited English Proficient (LEP) students include LEP students who 
took the English version of MCAS (primarily, LEP students who had been enrolled in school in the 
United States for more than three years), as well as eligible LEP students who took the Spanish- 
language version of MCAS. The Spanish-language version of MCAS included tests in Mathematics, 
Science 8c Technology, and History and Science only; English Language Arts tests were not included. 
Therefore, limited English proficient students who took the Spanish-language version of MCAS are 
shown as Other Not Tested on the English Language Arts page (page 3) of the School Report. 

The status of Student with Disabilities or Limited English Proficient is indicated separately for each 
subject area since discrepancies in the number of students in these categories across subject areas would 
result from inconsistent indications for a student across subject areas. 

Absent Students 

Columns 5 and 6 show the number and percentage of absent students who were reported absent during 
the testing period in spring 1999 (indicated by designated school personnel on Student Answer Booklets). 
Enrolled students for whom no answer booklets were returned were also defined as Absent. 

What is the impact of students who were Absent f Students from any student status group who were 
absent during the testing period, and who did not receive a medically documented absence, received a 
minimum score of 200, and by default, a performance level of Failing. Absent students are included in the 
determination of the scaled scores and performance level results for schools. This reporting decision was 
made to support the legal requirement that all students participate in the MCAS testing program. 

Some students reported as Absent on the Student Answer Booklet participated in one or more portions of 
a subject area test. In some instances, schools inadvertently marked Absent on the booklets of these stu- 
dents. In other cases, these students may have been present for some, but not all, portions of the test. 



22 



The Massachusetts Comprehensive Assessment System: 
Guide to Interpreting the 1999 MCAS Reports for Schools and Districts 



Given that these students would receive the minimum score of 200 by default, these students were 
designated as Tested and received their actual score (equal to or greater than 200) based on the 
portions of the subject area test in which they participated. 



Students Not Included in Scores 

Columns 7-10 of each table on pages 3-6 show the number and percentage of students who did not 
participate in the spring 1999 MCAS tests, and were reported as either Medically Documented 
Absent or Other Not Tested. 

Medically Documented Absent Students 

Students who obtained a written statement from a licensed medical doctor, stating both the medical 
reason that prevented them from taking the test and the dates for which they were absent due to the 
medical condition. 

What is the impact of students who were Medically Documented Absent f Medically Documented 
Absent students were not included in the determination of scaled score or performance level results. 

Other Not Tested 

Three categories of students are reported as Other Not Tested: 4 

(1) students with disabilities for whom the standard test was not appropriate even with 
accommodations and who participated in MCAS through a locally developed alternative 
assessment 

(2) limited English proficient (LEP) students who had been in school in the United States 
for three or fewer years and for whom the Spanish-language version of MCAS was not 
required in accordance with Department guidelines (designated as Other Not Tested for 
all subject area tests) 

(3) limited English proficient (LEP) students who had been in school in the United States for three 
or fewer years who participated in the Spanish-language version of MCAS and who, conse- 
quently, were designated as Other Not Tested for English Language Arts tests only since there 
is no Spanish version of the English Language Arts test. The Spanish-language version included 
tests for Mathematics, Science & Technology, and History and Social Science only; there was 
no English Language Arts section in the Spanish-language version of MCAS. 

What is the impact of students who were Other Not Tested? Since students who were Other Not 
Tested could not participate in the standard version of MCAS, average school scores are not affected 
by Other Not Tested students. Unlike students reported as absent, these students were not assigned 
minimum scores and were not included in computation of school and district averages. 



4 Additionally, there are a small number of cases where Student Answer Booklets or Composition Booklets could 
not be matched or students responded in more than one test booklet. For 1999 reports, these cases are also 
reported as Other Not Tested. 
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B. PERFORMANCE LEVEL RESULTS 

(REPORT PAGE 6 FOR GRADES 4 & 10; PAGES 7 FOR GRADE 8} 



Performance level results indicate, for each of the subject areas tested, the percentage of students 
performing at each of the four MCAS performance levels: Advanced, Proficient, Needs Improvement, 
and Failing. These percentages are based on the results of all tested students, including regular 
education students; students with disabilities (who participated with or without accommodations); 
and limited English proficient students (who participated in either the English or the Spanish- 
language version of MCAS). These results also include Absent students who received the minimum 
score (200/Failing), but do not include Medically Documented Absent or Other Not Tested students 
(defined above), for whom results were not generated. 



C. DISTRIBUTION OF RESULTS BY SCALED SCORE INTERVAL 

(REPORT PAGE 7 FOR GRADES 4 & 10; PAGE 8 FOR GRADE 8) 

These results show the distribution of student performance across 
the four MCAS performance levels — Advanced, Proficient , Needs 
Improvement, and Failing — for each of the subject areas tested. 

The distribution within each performance level is also shown across 
five score intervals. The percentage and number of students (at the 
school, district, and state levels) performing at each of these intervals 
are displayed. 

This table is designed to help schools see how students are clustered 
within each performance level and to measure movement within levels 
over time. It can also be used to illustrate how two or more schools 
with similar percentages of students at each performance level may have 
very different average scaled scores, resulting from different distributions 
of individual scores within performance levels. 
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D. RESULTS BY STUDENT STATUS 

(REPORT PAGES MO FOR GRAOES 4 & 10; PAGES 9-12 FOR GRAOE 8) 



The information on these report pages serves two primary purposes: to see how effectively school 
programs serve various student populations, and to promote fair comparisons between schools and 
between districts by allowing them to compare the performances of similar student groups. 

Subject-specific performance level results disaggregated by student subgroup 
can be found as follows (for student status definitions, see page 21 of this chapter): 

• Page 8: English Language Arts Results by Student Status 

• Page 9: Mathematics Results by Student Status 

• Page 10: Science & Technology Results by Student Status 

• Page 11: History and Social Science Results by Student Status (for Grade 8 only) 

Each page shows subject-specific results for the following student status groups: 

• All students (Total) 

• Regular education students (Regular) 

• Students with Disabilities 

• Limited English Proficient students 

Within each of these student status sections, pie charts graphically display the performance level 
results (percentage of students performing at each level) for the school, district, and state. The pie 
charts are shaded according to the key found in the table to the right. This table also shows the 
numbers of students at each performance level. Below each pie chart is the average scaled score for 
the school, district, and state, respectively. 
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E. SUBJECT AREA SUBSCORES 

(REPORT PAGES 1 1-12 FOR GRADES 4 & 10; PAGES 13-14 FOR GRADE 8) 



The purpose of these report sections is to help schools determine the extent to which their curriculum 
and instruction are effective in helping students achieve the standards contained in particular strands 
and substrands of the Curriculum Frameworks. 

Subject area subscores are the only reported MCAS scores based on both common and matrix- 
sampled items. Unlike common items, matrix-sampled items differ across test forms and are used 
to report performance on a broader range of specific curriculum goals than is possible by using 
common items only. 

Subject area subscores provide specific information on performance in various subdomains 
(“Reporting Categories”) of the tested subject areas, for example, Geometry and Measurement 
is a subdomain of grade 4 Mathematics. 5 In some subject areas, many fewer categories are included 
on the reports in 1999 than in 1998. In 1999, results have been collapsed into broader subdomains 
to minimize misinterpretation of the small differences in performance in reporting categories that are 
based on small numbers of items. For a complete listing of the 1999 MCAS reporting categories, see 
Appendix B. 

Information about each reporting category within each subject area tested includes 

• the number of possible points for that category on the common items, the matrix items, 
and the total number of possible points. 

• the average total number of points attained and the percentage of the total possible number 
of points attained at the school, district, and state levels. 6 

Subject area subscore results by subject area tested are provided on the following pages 
of the Report: 

• Page 11: English Language Arts 

• Page 11: Mathematics 

• Page 12: Science & Technology 

• Page 12: History and Social Science (Grade 8 only) 



Caution: Raw Scores versus Scaled Scores. The total number of possible points for a particular 
subject area reporting category is based on raw score points, which are not the same as scaled scores 
used elsewhere throughout the Report. Raw scores are the total number of points earned for all 
items administered in a particular subject area. Raw scores are converted to scaled scores through a 
data analysis process known as scaling. Scaling converts raw data points from one scale to another 
in a process that is analogous to the conversion of temperature in degrees Fahrenheit to degrees 
Celsius. See Appendix A for tables of the Raw Score to Scaled Score Conversions. 

5 All twelve forms of the test must have been administered in a school for accurate computation of subject area 
subscores. If fewer than 12 forms of the test were administered in a school, subject area subscores were not 
reported for that school. 

6 Because schools may have had somewhat different numbers of students taking each of the twelve test forms, 
average points were determined in a two-step process. First, the average raw score was determined for each 
unique item. Second, the sum of the average raw scores computed in step 1 was calculated. 
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Caution: Longitudinal Comparisons. It is not appropriate to compare performance on subject area 
subscores across years because the number of items contributing to each subscore is relatively small 
and the difficulty of the items may vary somewhat from year to year. This caution applies only to 
subject area subscores, not to performance level results and scaled scores. This is because perform- 
ance level results and scaled scores are based solely on common items; scores based on common 
items will be statistically equated from year to year. The pool of matrix-sampled items will vary from 
year to year as items move from the matrix-sampled sections to the common sections of the tests to 
replace publicly released common items. New items will be developed annually to supplement the 
matrix-sampled item pool. 



F. THREE-YEAR COMPARISON OF SCHOOL RESULT 

(REPORT PAGES 13-15 FOR GRADES 4 & 10; PAGES 15-18 FOR GRADE 8) 

In the future, schools will be able to make longitudinal comparisons of performance level results 
and scaled scores. These pages of the 1999 School Report reflect the results of the 1998 and 1999 
MCAS tests. Eventually, these data can be used to conduct longitudinal (three years of data) compar- 
isons of results for English Language Arts (page 13), Mathematics (page 14), Science & Technology 
(page 15), and History and Social Science (page 16). Note : for History and Social Science there were 
no scores in 1998 and there were results for grade 8 only in 1999. 
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Appendix A 

Raw Score to Scaled Score Conversions 




1999 MCAS Grade 4 Raw Score to Scaled Score Conversions 



Total Score 


English Language Arts 


Mathematics 


Science & Technology 




Scaled Score 


Scaled Score 


Scaled Score 


IV 


. 


- 


- 


71 


270 


- 


- 


70 


270 


- 


- 


69 ' 


268 


- 


- 


68 


268 


- 


- 


67 


266 


- 


- 


66 


264 


- 


- 


65 


264 


- 


- 


64 


262 


- 


- 


63 


258 


- 


- 


62 


256 


- 


- 


61 


256 


- 


- 


60 


254 


- 


- 


59 


252 


- 


- 


58 


250 


- 


- 


57 


250 


- 


- 


56 


248 


- 


- 


55 


246 


- 


- 


54 


244 


280 


280 


53 


242 


280 


280 


52 


- 242 


278 


280 


51 


240 


276 


278 


50 


238 


274 


276 


49 


238 


272 


274 


48 


236 


270 


272 


47 


236 


266 


270 


46 


236 


264 


268 


45 


234 


262 


266 


44 


234 


260 


264 


43 


232 


256 


262 


42 


230 


254 


262 


41 


230 


252 


260 


40 


228 


250 


258 


39 


228 


248 


256 



*One common multi plechoice item on the ELA test was not used in the calculation of any scores. Therefore the 
maximum possible score is 71 points. 
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1999 MCAS Grade 4 Raw Score to Scaled Score Conversions (continued) 



Total Score 


English Language Arts 
1 Scaled Score 


Mathematics 
Scaled Score 


Science & Technology 
Scaled Score 


38 


228 


244 


i 

254 


37 


226 


242 


252 


36 


226 


240 


250 


35 


224 


238 


248 


34 


224 


236 


246 


33 


222 


236 


244 


32 


222 


234 


242 


31 


220 


232 


240 


30 


220 


230 


240 


29 


220 


230 


238 


28 


218 


230 


236 


27 


218 


228 


234 


26 


216 


226 


232 


25 


216 


224 


230 


24 


214 


224 


230 


23 


214 


222 


228 


22 


214 


220 


226 


21 


214 


220 


224 


20 


212 


218 


222 


19 


212 


218 


220 


18 


210 


216 


220 


17 


210 


214 


218 


16 


208 


212 


216 


15 


208 


212 


214 


14 


206 


212 


212 


13 


206 


210 


210 


12 


206 


208 


208 


11 


204 


206 


206 


10 


204 


206 


206 


9 


204 


204 


204 


8 


200 


202 


204 


7 


200 


200 


202 


6 


200 


200 


200 


5 


200 


200 


200 


4 


200 


200 


200 


3 


200 


200 


200 


2 


200 


200 


200 


1 


200 


200 


200 


0 


200 


200 


200 



0 
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1999 MCAS Grade 8 Raw Score to Scaled Score Conversions 



Total Score 


English Language Arts 
Scaled Score 


Mathematics 
Scaled Score 


Science & Technology 
Scaled Score 


History and Social 
Science Scaled Score 


72 


268 


~ 


- 


- 


71 


268 


- 


- 


- 


70 


268 


- 


- 


- 


69 


266 


- 


- 


- 


68 


266 


- 


- 


- 


67 


262 


- 


- 


- 


66 


262 


- 


- 


- 


65 


260 


- 


- 


- 


64 


260 


- 


- 


- 


63 


258 


- 


- 


- 


62 


256 


- 


- 


- 


61 


256 


- 


- 


- 


60 


254 


- 


- 


- 


59 


254 


- 


- 


- 


58 


252 


- 


- 


- 


57 


250 


- 


- 


- 


56 


250 


- 


- 


- 


55 


248 


- 


- 


- 


54 


248 


274 


280 


280 


53 


248 


272 


276 


276 


52 


246 


270 


276 


274 


51 


244 


268 


274 


272 


50 


244 


266 


274 


268 


49 


242 


264 


111 


266 


48 


242 


262 


268 


264 


47 


240 


260 


266 


260 


46 


240 


258 


264 


258 


45 


238 


256 


262 


256 


44 


236 


254 


260 


252 


43 


236 


252 


256 


250 


42 


234 


252 


254 


248 


41 


234 


250 


252 


244 


40 


232 


248 


250 


242 


39 


230 


246 


248 


240 


38 


228 


244 


246 


238 


37 


228 


242 


242 


236 


36 


226 


242 


240 


234 


35 


224 


240 


238 


232 


34 


224 


238 


234 


232 


33 


222 


236 


232 


230 


32 


220 


234 


230 


228 
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1999 MCAS Grade 8 Raw Score to Scaled Score Conversions (continued) 



Total Score 


English Language Arts 
Scaled Score 


Mathematics 
Scaled Score 


Science & Technology 
Scaled Score 


History and Social 
Science Scaled Score 


31 


218 


232 


226 


226 


30 


218 


232 


224 


1 224 


29 


218 


230 


222 


224 


28 


216 


228 


218 


222 


27 


214 


226 


216 


220 


26 


212 


224 


214 


218 


25 


212 


222 


210 


218 


24 


210 


220 


208 


216 


23 


208 


220 


206 


214 


22 


208 


218 


206 


212 


21 


206 


216 


202 


210 


20 


204 


214 


200 


210 


19 


202 


212 


200 


208 


18 


202 


210 


200 


206 


17 


200 


208 


200 


204 


16 


200 


206 


200 


202 


15 


200 


204 


200 


202 


14 


200 


202 


200 


200 


13 


200 


200 


200 


200 


12 


200 


200 


200 


200 


11 


200 


200 


200 


200 


10 


200 


200 


200 


200 


9 


200 


200 


200 


200 


8 


200 


200 


200 


200 


7 


200 


200 


200 


200 


6 


200 


200 


200 


200 


5 


200 


200 


200 


200 


4 


200 


200 


200 


200 


3 


200 


200 


200 


200 


2 


200 


200 


200 


200 


1 


200 


200 


200 


200 


0 


200 


200 


200 


200 
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1999 MCAS Grade 10 Raw Score to Scaled Score Conversions 



Total Score 


English Language Arts 


Mathematics 


Science & Technology 




Scaled Score 


Scaled Score 


Scaled Score 


11 


280 


- 




71 


278 


- 


‘ 


70 


278 


- 




69 


274 


- 




68 


111 


- 




67 


270 


- 




66 


268 


- 




65 


266 


- 




64 


264 


- 




63 


262 


- 




62 


260 


- 




61 


oo 


- 




60 


256 


280 


280 


59 


256 


280 


280 


58 


252 


280 


280 


57 


252 


280 


280 


56 


250 


280 


280 


55 


248 


280 


278 


54 


246 


280 


276 


53 


244 


280 


274 


52 


244 


278 


111 


51 


240 


276 


111 


50 


240 


274 


270 


49 


238 


272 


268 


48 


236 


268 


266 


47 


236 


266 


264 


46 


232 


264 


262 


45 


232 


264 


262 


44 


230 


260 


258 


43 


228 


258 


256 


42 


226 


256 


254 


41 


224 


254 


252 


40 


224 


250 


252 


39 


222 


248 


250 


38 


220 


248 


248 


37 


218 


246 


244 


36 


216 


244 


242 


35 


214 


242 


242 


34 


212 


240 


240 


33 


210 


236 


238 


32 


210 


234 


236 


31 


208 


232 


234 
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1999 MCAS Grade 10 Raw Score to Scaled Score Conversions (continued) 



Total Score 


English Language Arts 
Scaled Score 


Mathematics 
Scaled Score 


Science & Technology 
Scaled Score \ 


30 


206 


230 


232 


29 


t 204 


228 


230 ! 


28 


202 


226 


228 


27 


202 


226 


228 


26 


200 


224 


226 


25 


200 


222 


224 


24 


200 


220 


220 


23 


200 


218 


220 


22 


200 


216 


218 


21 


200 


214 


216 


20 


200 


212 


214 


19 


; 200 


210 


214 


18 


! 200 


208 


212 


17 


200 


206 


210 1 


16 


200 


202 


208 


15 


200 


200 


208 


14 


200 


200 


204 


13 


200 


200 


202 


12 


200 


200 


200 


11 


200 


200 


200 


10 


200 


200 


200 


9 


200 


200 


200 


8 


200 


200 


200 


7 


200 


200 


200 


6 


200 


200 


200 


5 


200 


200 


200 


4 


200 


200 


200 


3 


200 


200 


200 


2 


200 


200 


200 


1 


! 200 


200 


200 


0 


200 


200 


200 ! 
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Calculating Threshold Scores 



In 1998, in order to determine the minimum total test score (threshold score) for each performance 
level on each English Language Arts, Mathematics, and Science & Technology test, panelists exam- 
ined sets of student work in a standard-setting process. The standards established for these subject 
areas in 1998 remain unchanged for the 1999 MCAS tests. However, because the common test items 
on the 1999 MCAS tests are different from the common items on the 1998 test, the threshold scores 
representing those standards may change. In part, a change in threshold scores may be due to differ- 
ences in difficulty between the 1998 and 1999 MCAS test items. To some extent, however, the 
change in threshold scores is simply due to the changes in the number of multiple-choice and open- 
response items, and the change in the maximum possible score on most tests. Using the matrix-sam- 
pled items which remained unchanged from 1998 to 1999, the 1998 and 1999 MCAS tests are 
linked through a process called equating. Based on the equating process, the adjustments in the 
threshold scores needed to maintain the 1998 standards on the 1999 MCAS tests are determined. 

The eighth grade History and Social Science test was administered for the first time in 1999. In 
August 1999, threshold scores for each performance level on the History and Social Science test were 
established at a standard-setting meeting. The procedures used to establish the threshold scores for 
this test were consistent with the standard-setting procedures used to establish threshold scores for 
the 1998 MCAS tests. 



Threshold Scores, MCAS Tests of May 1999 



Threshold Score Between Levels 



Grade 


Subject Area 


Maximum 

Score* 


Advanced/ 

Proficient 


Proficient/ 

Needs 

Improvement 


Needs 

Improvement/ 

Failing 




English Language Arts 


71 


64 


51 


29 


4 


Mathematics 


54 


44 


36 


21 




Science &C Technology 


54 


41 


30 


18 




English Language Arts 


72 


64 


46 


32 


8 


Mathematics 


54 


47 


35 


23 




Science &C Technology 


54 


44 


36 


29 




History and Social Science 


54 


47 


39 


27 




English Language Arts 


72 


62 


50 


38 


10 


Mathematics 


60 


44 


34 


24 




Science &C Technology 


60 


45 


34 


23 



* The Maximum Score is the total number of points that a student could earn in the subject area if he/she 
answered all of the multiple-choice questions correctly (one point per question) and received the highest possible 
scores for all open-response questions, short-answer questions, and writing compositions. 
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Appendix B 

MCAS 1999 Reporting Categories 



MCAS reporting categories used in 1998 were collapsed in 1999 to discourage the misinterpretation 
of results based on the small numbers of questions upon which some 1998 reporting categories were 
based. When interpreting test results, however, it is still important to have a clear understanding of 
the content covered by the test questions. The tables in this Appendix show the number and the per- 
centage of points on the entire test (common and matrix items) within each of the 1999 reporting 
categories. A check mark (/) signifies that this subarea is included within the reporting category. 

For further information on reporting categories and their related assessment expectations, refer 
to the publication, MCAS Guide to the Massachusetts Comprehensive Assessment System for each 
content area. 

English Language Arts 
Grade 4, 8 and 10: 

Language 

Literature 

Composition: Topic Development 
Composition: Writing Conventions 

Mathematics 
Grade 4, 8 and 10: 

Number Sense 

Patterns, Relations, and Functions 
Geometry and Measurement 
Statistics and Probability 

Science & Technology 
Grade 4, 8 and 10: 

Inquiry 

Physical Sciences 
Life Sciences 

Earth and Space Sciences 
Technology 

History and Social Science 
Grade 8 and 10: 

U.S. History (Grade 8 only) 

World History 

Geography 

Economics 

Civics and Government 



o 

ERLC 
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English Language Arts 





Grade 4 


Grade 8 


Grade 10 




Points 


Percent 


Points 


Percent 


Points 


Percent 


Language 


36 


11.6% 


29 


9.3% 


33 


10.6% 


Literature 


255 


82% 


263 


84.3% 


259 


83.0% 


Composition * 


20 


6.4% 


20 


6.4% 


20 


6.4% 


Topic Development 


/ 




/ 




/ 




Conventions 


/ 




/ 




/ 




TOTAL 


311 


100% 


312 


100% 


312 


100% 



* There was one unique common prompt for Composition administered at grades 4, 8, and 10 in 1999. 
Students could earn up to 6 points for Topic Development, and up to 4 points for Use of Standard English 
Conventions (a total of 10 points). Each composition was scored twice (once by each of two different scorers) 
and the scores were combined. Therefore, a student could earn up to a maximum of 20 points on the 
Composition section of the MCAS. 
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Mathematics 





Grade 4 


Grade 8 


Grade 10 


Reporting Category 


Points 


Percent 


Points 


Percent 


Points 


Percent 


Number Sense 


68 


34.3% . 


50 


25.3% 


41 


20.1% 


Number Sense and Numeration 


/ 












Concepts of Whole Numbers 


/ 












Fractions and Decimals 


1 

/ 












Estimation 


/ 












Whole Number Computation 


/ 












Number and Number Relationships 






/ 








Number Systems, Number Theory 






/ 








Computation, Estimation 






/ 








Ratio, Proportion, Percent 






/ 








Discrete Mathematics 


1 








/ 




Mathematical Structure 










/ 




Estimation 










/ 




Patterns, Relations, and Functions 


41 


20.7% 


59 


29.8% 


65 


31.9% 


Patterns and Relations 


/ 












Algebra and Mathematical Structure 


/ 












Patterns and Functions 






/ 








Algebra 






/ 




/ 




Functions and Relationships 










/ 




Trigonometry 










/ 




Geometry and Measurement 


52 


26.3% 


50 


25.3% 


61 


29.9% 


Geometry and Spatial Sense 


/ 








/ 




Measurement 


/ 




/ 




/ 




Geometry 






/ 








Geometric Measurement 






/ 








Geometry from an Algebraic Perspective 










/ 




Statistics and Probability 


37 


18.7% 


39 


19.7% 


37 


18.1% 


Statistics 






/ 




/ 




Probability 






/ 




/ 




TOTAL 


198 


100% 


198 


100% 


204 


100% 
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Science & Technology 





Grade 4 


Grade 8 


Grade 10 




Points 


Percent 


Points 


Percent 


Points 


Percent 


Inquiry 


36 


19.4% 


29 


15.7% 


14 


6.9% 


Classification 


/ 












Designing an Investigation 


/ 




/ 




/ 




Data Collection, Measurement, and Display 


/ 




/ 








Analysis and Interpretation of Data 


/ 




/ 




/ 




Physical Sciences 


37 


19.9% 


34 


18.4% 


, 52 


25.5% 


Properties of Matter 


/ 




/ 








Structure of Matter 










/ 




Interactions of Substances 










/ 




Position and Motion of Objects 


/ 








/ 




Motion 






/ 








Energy 


/ 




/ 




/ 




Life Sciences 


40 


21.5% 


35 


18.9% 


49 


24.0% 


Characteristics of Organisms 


/ 




/ 




/ 




Adaptations, Diversity, and Heredity 


/ 












Diversity, Adaptation, and Reproduction 






/ 




< 




Heredity and Evolution 










/ 




Organisms and Environments 


/ 












Ecosystems and Organisms 






/ 








Ecosystems 










/ 




Earth and Space Sciences 


37 


19.9% 


41 


22.2% 


48 


23.5% 


Properties of Earth's Materials 


/ 








: 




Interactions/Cycles in the Earth System 






/ 








Earth's Processes 










/ 




Objects in the Sky 


/ 












Earth and Space 






/ 








Solar System and Universe 










/ 




Technology 


36 


19.4% 


46 


24.9% 


41 


20.1% 


The Design Process 


/ 




/ 




/ 




Understanding and Using Technology 


/ 




/ 




/ 




Science, Technology, and Human Affairs 


/ 




/ 




/ 




TOTALS 


186 


100% 


185 


100% 


' 204 


100% 
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History and Social Science 





; Grade 8 


Grade 10 




Points 


Percent 


Points 


Percent 


History: 










World 


! 29 


15.7% 


99 


47.4% 


United States 


1 51 


27.6% 






Chronology and Cause 


/ 




/ 




Historical Understanding 


/ 




/ 




Research, Evidence and Point of View 


/ 




/ 




Society, Diversity, Commonality and the Individual 


f 

/ 




/ 




Interdisciplinary Learning: Religion, Ethics, 
Philosophy and Literature 


/ 




/ 




Interdisciplinary Learning: Natural Science, 
Math and Technology 


/ 




/ 




Geography 


38 


20.5% 


31 


14.8% 


Physical Spaces of the Earth 


/ 




/ 




Places and Regions of the World 


/ 




/ 




Effects of Geography 


! / 




/ 




Human Alteration of the Environment 


! / 




/ 




Economics 


34 


18.4% 


29 


13.9% 


Fundamental Economic Concepts 


/ 




/ 




Economic Reasoning 


/ 




/ 




American and Massachusetts Economic History 


/ 




/ 




Today's Economy 


/ 




/ 




Theories of Economy 


/ 




/ 




Civics and Government 


33 


17.8% 


50 


23.9% 


Authority, Responsibility and Power 


/ 




/ 




Founding Documents 


/ 




/ 




Principles and Practices 


/ 




/ 




Citizenship 


/ 




/ 




Forms of Government 


/ 




/ 




TOTAL 


185 


100% 


209 


100% 
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Appendix C 

Content-Specific Performance Level Definitions 
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GO 




42 
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Student work at the Failing level (200-219) does not meet the criteria of the Needs Improvement level. 
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Appendix D 

MCAS Committee Members, 1996-1999 



ENGLISH LANGUAGE ARTS ASSESSMENT DEVELOPMENT COMMITTEE 



Bill Amorosi 
Sorel Berman 
Ann Connolly-Tolkoff 
Anne Graham 
Yvonne Gunzburger 
Susan Horn 
William Irvin 
Shirley Kountze 
James McDermott 
Laurie Palmer 
Lorraine Plasse 
David Roach 
Anne Steele 
Sandra Stotsky 
George Viglirolo 
Robert Zeeb 



Belmonte Middle School, Saugus Public Schools 
Brookline High School, Brookline Public Schools 
City on a Hill Charter School (Boston) 

Galvin Middle School, Wakefield Public Schools 
Hemenway Elementary, Framingham Public Schools 
Adams-Cheshire Regional Public Schools 
Pittsfield Public Schools 

Brooks/Hobbs Magnet School, Medford Public Schools 
South High Community School, Worcester Public Schools 
Memorial School, Natick Public Schools 
Springfield Public Schools 
Millbury Public Schools 

Shrewsbury High School, Shrewsbury Public Schools 
Harvard Graduate School of Education/Boston University 
Brookline High School, Brookline Public Schools 
Newton Public Schools 



MATHEMATICS ASSESSMENT DEVELOPMENT COMMITTEE 



Jim Alberque 
Brian Barnes 
Peg Bondorew 
Maureen Chapman-Fahey 
David Daniels 
William Day 
Hal Dickert 
Paul Donovan 
Barbara Haig 
Marcia Harol 
Maggi Hartnett 
Patricia Hills 
Carol Hynes 
Joan Kenney 
Deborah King 
Michele Kingsland-Smith 
Raynold Lewis 
Gloria Moran 
Donna Pappalardo 
Christine Redford 



Worcester State College 

Mansfield High School, Mansfield Public Schools 
Northeastern University 
Medford Public Schools 

Longmeadow High School, Longmeadow Public Schools 
Lawrence School, Falmouth Public Schools 
Hopkinton Middle School, Hopkinton Public Schools 
Blue Hills Regional Technical High School 
Marion Zeh School, Northborough Public Schools 
Andover High School, Andover Public Schools 
Ayer Senior High School, Ayer Public Schools 
Holyoke Public Schools 

Leominster High School, Leominster Public Schools 
Harvard Graduate School of Education 
Monatiquot School, Braintree Public Schools 
Ahern Middle School, Foxborough Public Schools 
Worcester Technical Institute 

M.G. Williams Junior High, Bridgewater-Raynham School District 
Parker Middle School, Reading Public Schools 
Joshua Eaton Elementary, Reading Public Schools 
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Guy Roy 
Bernard Ryder 
Donna Scanlon 
Margaret Skowron 
Nancy Sprague 
Kathy VanCamp 
Nancy Zamarro 
Giselle Zangari 



Plymouth Public Schools 
Agawam Public Schools (Retired) 

Holyoke Public Schools 

Highland Elementary School, Brimfield Public Schools 
Bridgewater State College 

Brimfield Elementary School, Brimfield Public Schools 
Worcester Vocational High School, Worcester Public Schools 
Boston University Academy 



SCIENCE & TECHNOLOGY ASSESSMENT DEVELOPMENT COMMITTEE 



Althea Brown 
Kathleen Brown 
Paul Cavanagh 

Mary Corcoran 
Charles Corley 
Mary Creed 
Joyce Croce 
Howard Dimmick 
Susan Ferguson-Ellia 
John Fusco 
Bradford George 
Ilia Gonzalez Alonso 
Diane Goodman 
James Hamos 
Michael Lewandowski 
Michael Lynch 
Patrick Markham 
Maureen Moir 
Louise Mary Nolan 
Maxine Rosenberg 
Robert Sartwell 
Peter Shaughnessy 
Pamela Tickle 
Maria Torres 
Mike Zapantis 



Medford High School, Medford Public Schools 
Hudson Public Schools 

North Attleborough High School, North Attleborough 
Public Schools 

Massachusetts Association of Science Supervisors 

McCall Middle School, Winchester Public Schools 

Fall River Public Schools 

Tyngsborough Public Schools 

Stoneham Public Schools 

Oxford Middle School, Oxford Public Schools 

Winchester High School, Winchester Public Schools 

Hale Middle School, Stowe Public Schools 

Cambridge Public Schools 

Alfred Zanetti School, Springfield Public Schools 

University of Massachusetts Medical Center 

Joseph Case High School, Swansea Public Schools 

Andover High School, Andover Public Schools 

Pittsfield Public Schools 

Bridgewater State College 

Woburn Public Schools 

Newton Public Schools 

Agawam Public Schools 

Northampton High School, Northampton Public Schools 
FallRiver Public Schools 
Fitchburg Public Schools 
Pittsfield Public Schools 



HISTORY AND SOCIAL SCIENCE ASSESSMENT DEVELOPMENT COMMITTEE 



June Coutu 

Daniel Dodson 
Jim Dudley 
Alan Fraker 
Susan Goldsmith 
Roberta Logan 
Joyce Malcolm 
Beverly Nelson 



King Philip Regional High School, King Philip 
Regional School District 

Marsh Grammar School, Methuen Public Schools 

Brookline High School, Brookline Public Schools 

Boston University School of Education 

Boston University School of Education 

Martin Luther King Middle School, Boston Public Schools 

Bentley College 

Medford Public Schools 
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Marie Norris 
Sheldon Stern 
Susan Szachowicz 
Fred Wetzel 
Bernadette Wilkinson 



McKay School, Beverly Public Schools 

JFK Library and Museum 

Brockton High School, Brockton Public Schools 

The College Board 

Lincoln School, Winchester Public Schools 



STATEWIDE ASSESSMENT ADVISORY COMMITTEE 



James Argir 
Guessippina Bonner 
MaryAnn Byrnes 
Mary Campbell 
Jim Caradonio 
John Cawthorne 
John Collins 
Kathleen Conole 
Ruth Ann Corbin 
June Coutu 
Maryellen Donahue 
David Fredette 
A1 Galante 
Lorraine G re iff 
Ellen Guiney 
William Irvin 
Julia Landau 
Yu-Lan Lin 
Charles E. Martin, Jr. 
Louise Mary Nolan 
Stephen H. Pronovost 
F. Paul Quatromoni 
Angel G. Ramirez, Jr. 
Jonathan Rappaport 
Jack Rennie 
Paul Reville 
Roger L. Rice 
Dennis Richards 
Connie Rizoli 
Gregory T. Scotten 
Frank Vacirca 
Brendan Walsh 



Massachusetts Elementary School Principals Association 

Massachusetts Teachers Association 

Consultant on Special Education 

Horace Mann School for the Deaf, Boston Public Schools 

Worcester Public Schools 

School of Education, Boston College 

Holy Cross College 

Greater Lowell Regional Vocational Technical School 
Massachusetts Vocational Association 
Massachusetts Council for the Social Studies 
Boston Public Schools 

Massachusetts Council of Teachers of English 
Massachusetts Association of Teachers of Mathematics 
Massachusetts Office on Disability 
Boston Plan for Excellence 
Pittsfield Public Schools 
Massachusetts Advocacy Center 
Massachusetts Foreign Language Association 
Rockport Public Schools 

Massachusetts Association of School Superintendents 
Massachusetts Secondary School Administrators Association 
Massachusetts Association of Science Supervisors 
Massachusetts Association of School Committees 
Worcester Public Schools 
Massachusetts Business Alliance 
Harvard Graduate School of Education 
Multicultural Education Training and Advocacy 
Reading Public Schools 

House Committee on Education, Massachusetts State Legislature 
Massachusetts Secondary School Administrators Association 
Massachusetts Association of Vocational Administrators 
Council of Administrators of Compensatory Education 



ENGLISH LANGUAGE LEARNER FOCUS GROUP 



Bethel Bilezikian Charkoudian 

Mary Cazabon 

Marguerite Goes 

Georgette Gonsalves 

Mary Ann Lachat 

Jill McCarthy 

Susan J. McGilvray-Rivet 



English High School, Boston Public Schools 

Cambridge Public Schools 

Lowell Public Schools 

Boston Public Schools 

The Regional Lab, Brown University 

Newton Public Schools 

Framingham Public Schools 
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Marla Perez-Selles 
Kay Polga 
Rosalie Porter 
Roger Rice 
Kathryn L. Riley 
Connie Rizoli 



Cambridge Public Schools 
Brookline Public Schools 
READ Institute 

Multicultural Education Training and Advocacy 
Consultant on Bilingual Education 

House Committee on Education, Massachusetts State Legislature 



SPECIAL EDUCATION FOCUS GROUP 

Michael Bello 
Mary Ann Byrnes 
Joan DeGeorge Schirmer 
Cynthia Essex 
Julia Landau 
Katherine Levine 
William H. Marginson 
Tom Miller 

Lorna Nickerson Kaufman 
Suzanne Recane 
David Riley 
Richard Robison 
Tim Sindelar 
Joanne Testaverdi 



Learning Center for Deaf Children 
Consultant on Special Education 
West Bridgewater Public Schools 
Perkins School for the Blind 
Massachusetts Advocacy Center 
East Bridgewater Public Schools 
New Bedford Schools 
Perkins School for the Blind 
Kaufman Associates 
Learning Center for Deaf Children 
Massachusetts Urban Project 
Federation for Children 
Disability Law Center 
Northeast Regional Vocational School 



BIAS REVIEW COMMITTEE MEMBERS 


Anthony Baxter 


Salem State College 


Gwenn Blackburn 


Medford High School, Medford Public Schools 


Guessippina Bonner 


Massachusetts Teachers Association 


Cathleen Boynton 


Brockton High School, Brockton Public Schools 


Althea Brown 


Medford High School, Medford Public Schools 


Kriner Cash 


Vineyard Haven Public Schools 


Kerry Cavallaro 


Norfolk County Regional Vocational High School 


John Cawthorne 


School of Education, Boston College 


Veronica Griffin 


Worcester Public Schools 


Sumru Erkur 


Stone Center, Wellesley College 


Carol House 


Cambridge Rindge and Latin School, Cambridge Public Schools 


Deidre Loughlin 


Worcester Public Schools 


Paula Martin 


Needham Public Schools 


Fern Marx 


Stone Center, Wellesley College 


Margarita Poles 


Parent, Boston Public Schools 


Lionel Rein ford 


Worcester Public Schools 


Wanda S. Franklin 


Needham Public Schools 


Meg Wilder Watson 


Boston Public Schools 
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NATIONAL TECHNICAL ADVISORY COMMITTEE 



Ron Hambleton 
George Madaus 

Barbara Plake 
Doug Rindone 
Roger Trent 



University of Massachusetts, Amherst 
Center for the Study of Testing, Evaluation and 
Educational Policy, Boston College 
University of Nebraska 
Connecticut Department of Education 
Ohio Department of Education 






The Massachusetts Comprehensive Assessment System: 

Guide to Interpreting the 1999 MCAS Reports for Schools and Districts 



Appendix E 






MCAS-Related References 



MASSACHUSETTS CURRICULUM FRAMEWORKS 

Massachusetts Department of Education, English Language Arts Curriculum Framework 
(Malden: 1997). 

Massachusetts Department of Education, Mathematics Curriculum Framework: Achieving 
Mathematical Power (Malden: 1996). 

Massachusetts Department of Education, Science & Technology Curriculum Framework: 
Owning the Questions Through Science & Technology (Malden: 1996). 

Massachusetts Department of Education, Flistory and Social Science Curriculum Framework 
(Malden: 1997). 



MCAS GUIDES 

Massachusetts Department of Education, Guide to the Massachusetts Comprehensive Assessment 
System: English Language Arts (Malden: 1998). 

Massachusetts Department of Education, Guide to the Massachusetts Comprehensive Assessment 
System: Mathematics (Malden: 1998). 

Massachusetts Department of Education, Guide to the Massachusetts Comprehensive Assessment 
System: Science & Technology (Malden: 1998). 

Massachusetts Department of Education, Guide to the Massachusetts Comprehensive Assessment 
System: Flistory and Social Science (Malden: 1999). 



OTHER MCAS DOCUMENTS 

Massachusetts Department of Education, The Massachusetts Comprehensive Assessment System: 

Requirements for the Participation of Students with Disabilities (A Guide for Educators and 
Parents) (Malden: 1999). 

Massachusetts Department of Education, The Massachusetts Comprehensive Assessment System: 

Release of Spring 1999 Common Items (Malden: 1999). 

Massachusetts Department of Education, The Massachusetts Comprehensive Assessment 
System: Requirements for Test Scheduling, Student Participation, and Test Security and Ethics 
(Malden: 1998). 
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